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ABSTRACT 

This paper reports the findings of a case study in which audio feedback replaced text-based feedback in 
asynchronous courses. Previous research has demonstrated that participants in online courses can build 
effective learning communities through text based communication alone. Similarly, it has been 
demonstrated that instmctors for online courses can adequately project immediacy behaviors using text- 
based communication. However, we believed that the inclusion of an auditory element might strengthen 
both the sense of community and the instmctor’s ability to affect more personalized communication with 
students. Over the course of one semester, students in this study received a mixture of asynchronous audio 
and text-based feedback. Our findings revealed extremely high student satisfaction with embedded 
asynchronous audio feedback as compared to asynchronous text only feedback. Four themes, which 
accounted for this preference, were culled out in an iterative, inductive analysis of interview data: 1. 
Audio feedback was perceived to be more effective than text-based feedback for conveying nuance; 2. 
Audio feedback was associated with feelings of increased involvement and enhanced learning community 
interactions; 3. Audio feedback was associated with increased retention of content; and 4. Audio feedback 
was associated with the perception that the instructor cared more about the student. Document analysis 
revealed that students were three times more likely to apply content for which audio commenting was 
provided in class projects than was the case for content for which text based commenting was provided. 
Audio commenting was also found to significantly increase the level at which students applied such 
content. Implications of this case study and directions for future research are addressed in the discussion 
and conclusions section of this paper. 
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I. INTRODUCTION 

As the number of online courses continues to expand, so must the ways in which instructors engage in 
active facilitation of learning among their students. This study focuses on one aspect of facilitation, the 
way in which we communicate and guide students in asynchronous learning networks (ALN) and how 
this process might be improved upon. 


While the evolution of ALN has made it increasingly easier to involve remotely based students in two- 
way communications [1] and enable students to process more complex information [2], instructors are 
often required to adapt to new roles [3]. While several frameworks have been developed to explain the 
role of the instructor [3, 4, 5], a system first proposed by Berge [6] and later refined by others [7] 
proposes a four part model consisting of pedagogical, social, technical, and managerial dimensions, each 
with a varying number of roles. For purposes of this study, the social dimension and three roles 
(profession-inspirer, feedback-giver, and interaction-facilitator) within the pedagogical dimension are 
considered the most important. These are depicted in the following table which was derived from work by 
Liu and colleagues [2]. 


Dimensions 

Roles 

Description of Roles 

Pedagogical 

Profession-inspirer 

Promote professional dialogue among online learners; relate 
personal experiences and cases to the discipline; point to 
professional organizations. 


Feedback-giver 

Provide timely and high quality feedback; provide 
formative feedback for continuous learning engagement. 


Interaction- facilitator 

Facilitate peer interaction in online discussion through a 
wide range of facilitation strategies. 

Social 

Social rapport builder 

Build social rapport; establish online teams; build online 
learning community. 


Table 1. Select Roles of Online Instructors 


In the traditional face-to-face classroom setting, each of these roles would be dependent upon both verbal 
and non-verbal cues. In the online environment, however, the primary form of communication is via text 
and therefore devoid of traditional paralinguistic cues [2]. Arbaugh [8] suggests that the relative low 
richness of text-based communication may make interdependent, ambiguous tasks particularly 
challenging. 


Critics of online learning, building on the low richness of text-based communication, contend that 
because interactions occur in a disembodied form, this lack of nuance leads to a loss of meaning [9, 1 0, 
11, 12]. As such, it is argued that asynchronous learning is not sufficiently rich in the socially mediated 
practice that Vygotsky [13] described as necessary to construct knowledge. However, this narrow 
interpretation of Vygotsky discounts the ability of learners to conceptualize “being” as anything other 
than a physical construct. 
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The ability to project oneself through various media, termed social presence, was initially described by 
Short, Williams and Christie [14] who proposed that, as critics of asynchronous learning contend, the 
ability to project verbal and nonverbal information directly impacted the degree to which presence was 
perceived. Flowever, Rourke, Anderson, Garrison and Archer [15] and Swan [16] argued that this may not 
be the case as learners in online courses appeared to build effective learning communities by projecting 
their personalities through text alone. 


Lombard and Dutton [17] viewed this creation of a presence in online courses as the ability to project 
oneself into a virtual. In an extension of this concept, Laffey, Lin and Lin [18] described the social 
element of asynchronous communication evolving as learners come to view their interactions with tasks 
and tools as being a fluid, integrated process rather than as a series of tasks. They compared this process 
to a speaker interacting with others in a foreign language. The more fluent the speaker becomes with the 
new language the less difficult interactions become. Theoretically, this would mean that the technologies 
become part of the interaction itself and are therefore not viewed as objects upon which learners have to 
act to create virtual embodiments [19]. 


Gunawardena and Zittle [20] found that the sense of “being there” was established in the online 
environment through providing and interpreting emoticons as a replacement for nuance and nonverbal 
cues. Using a 14-item questionnaire, they found 60% of the variance in student satisfaction was 
attributable to perceptions of social comfort and presence. Rovai [21] explained that this type of 
satisfaction can occur when text based, socio-emotional-driven interactions promote a sense of 
connectedness among learners in asynchronous learning networks (ALN). 


Richardson and Swan [22] used regression analyses to determine the relationship between perceived 
social presence and perceived learning. Analysis of data collected from 1 7 courses revealed that 46% of 
the variability in perceived learning could be predicted by student perceptions of social presence. 
However, the study also revealed that an even stronger relation (R^ = 0.53) existed between perceived 
learning and overall satisfaction with the instructor. This finding indicated that satisfaction with the 
instructor was at least as important as was perceived social presence. Further, the authors found that a 
strong relation (R^ = 0.36) existed between students’ perceptions of social presence and satisfaction with 
the instructor. Based on these findings it was concluded that “students’ perceptions of social presence 
were related to the perceptions of their instructors as having a satisfactory online presence in terms of 
amount of interaction and/or quality of that interaction.” 


Through factor analysis, Arbaugh [23] found instructor immediacy behaviors in online courses were a 
significant predictor of student learning. Based on Gorham’s [24] verbal immediacy scale, Arbaugh 
defined immediacy behaviors as being comprised of two parts. The first, classroom demeanor, “reflected 
the instructor’s use of personal examples, humor, and openness toward and encouragement of student 
ideas and discussion.” The second, name recognition, referred to the “extent to which the instructor was 
addressed by name by the students and vice versa.” 


A. Instructional Design Features that Foster Community 

Informed by the studies previously discussed, we have been improving on our design of ALN instruction 
to facilitate meaningful discourse and create dynamic learning environments. Specifically, in our courses 
over the past six semesters, we have attempted to incorporate recommendations found in the literature 
related to the projection of teaching presence through immediacy behaviors. Surveys of student 
satisfaction from these courses indicated that students were generally highly satisfied with our efforts and 
students’ qualitative feedback, when provided, typically made us believe we were doing a good job of 
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creating a rich learning environment. Flowever, even if social presence is strong, student may prefer even 
more interactive communication [25, 26], and we question whether greater interaction might also apply to 
the projection of teaching presence. Arbaugh [8] found media variety to be positively associated with 
perceived learning among students in web-based MBA courses. In a review of the literature, Liaw and 
Haung [27] suggested that presentation of web-based course content through a variety of media positively 
impacted learner experiences. 


Ideally, we would have liked to have used an asynchronous videoconferencing mechanism similar to that 
envisioned by Watt, Walther and Nowak [28]. Extending work by Walther and Burgoon [29], Watt and 
colleagues wrote that such a system would take full advantage of both verbal and nonverbal cues thereby 
increasing copresence; “the sense that one is actively being perceived and that one is actively perceiving 
another [28].” However, based on previous student surveys we knew that approximately one third of our 
students were likely to be taking classes via dialup connections, making the use of streaming video 
impractical. Thus, the only feasible alternative available to us was the use of asynchronous audio. 


Research on the use of stand alone audio in AEN, especially audio feedback, is rather limited. The study 
that provided us with the most insight as to how audio feedback might be perceived by students was 
conducted by Jelfs and Whitelock [30]. These researchers created a virtual environment in which various 
navigational techniques were used. All of the participants indicated in follow-up interviews that the 
preprogrammed auditory feedback was as important to their success and satisfaction with the environment 
as was ease of navigation. Significantly, these two factors were considered to be even more important 
than interactivity or previous experience. 


B. Use of Audio Feedback 

Use of audio commenting in the face-to-face classroom can be traced to at least 1982, when Olson [31] 
reported using the technique in English courses at a two year college. In a discussion of the technique, 
Olson opined that his students believed audio commenting reflected a sense of caring on the part of the 
instructor that extended beyond their written products. The ability to project through tone of voice, he 
argued, enabled the instructor “to be more supportive and caring.” 


Building on Olson’s work, Mellen and Summers [32] provided students in an English course with tapes 
containing audio feedback and conducted surveys and interviews at the end of the semester. Results 
demonstrated that students were likely to view audio feedback as being positive regardless of the context. 
Additionally, 70% of students reported that they felt encouraged to revise their work as a result of 
receiving auditory feedback and 54% felt more confident about their writing. These findings provide 
strong, highly positive indicators of student perceptions regarding the use of audio feedback and point to 
its potential as a tool in asynchronous online courses. 


In a study of student-student audio based interactions in AEN, Kim [33] found that students had generally 
positive perceptions of the medium, but that its use decreased motivation. However, audio did increase 
social presence, a finding that supported earlier research in which Reeves and Nass [34] concluded that 
human voice increased social presence. In a seeming contradiction, Bargeron and colleagues [34] found 
that students preferred to use text rather than audio in threaded discussions because they found it easier 
and quicker to read text messages than listen to audio. 


However, the sample size in the study conducted by Bargeron and colleagues [35] was small with only 4 
of the 6 total participants indicating a preference for text based feedback. We conducted a pilot study 
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asking 83 of our students to complete a survey regarding the relative time required to utilize text based 
versus audio feedback. We found that 28 students believed it took longer to listen to audio feedback than 
to read text-based feedback, 35 believed the time required was approximately the same, and 20 believed it 
took less time to listen to audio feedback. In addition, after answering questions about the time required to 
listen to audio feedback, 6 students emailed the instructor wishing to clarify their answers. The following 
is representative of the emails received: 

I just finished answering some questions about the time it took to listen to comments or read 
comments. My answer was that it took longer. However, I wanted to clarify that a little. It took 
longer because I replayed the co mm ents a couple of times so I could really see what was being 
said as it related to my work and get more out of it. I don’t do this when the comments are written 
because I don’t think they are as good. 


Based on these findings, we concluded that the difference in time required to listen to audio feedback 
versus reading text-based feedback was not a significant factor in deciding whether the technique should 
be used. In fact, based on the supplemental feedback, there was reason to believe that even though some 
students perceived audio feedback to be more time consuming, they still preferred it because they 
believed they got more out of it. Clearly, more research is needed in this area to explore students’ 
perceptions related to each type of feedback. 


The research clearly shows connections between perceived learning, perceptions of social presence, 
instructor satisfaction, and immediacy behaviors in building a sense of community among ALN learners. 
Y et to be established, however, is the extent to which auditory feedback might further enhance teaching 
presence and therefore build a stronger student sense of community. 


II. METHOD 

From spring 2004 through summer 2005, we served as instructors in seven asynchronous online courses. 
Despite being highly satisfied with the experiences and believing that our students had significant 
learning experiences, we wondered if we had done all we could to make our relationships with students as 
personal as possible given the constraints of the medium. While we disagree with those who view online 
learning as detached and impersonal [36, 37], we were concerned about our ability to adequately convey 
nuance in a manner similar to that which occurs in face-to-face classrooms. This concern prompted our 
research to better understand the nature of audio feedback in an asynchronous learning network. 
Specifically, in this study we sought to answer the following set of research questions (RQ): 

• RQ 1 : Between audio and text-based student feedback in ALN, which do students believe is a 
more effective means of interaction with their instructor? 

• RQ 2: To what degree do students believe audio feedback is an effective replacement of 
instructor/student interaction that typically occurs in traditional face-to-face classes? 

• RQ 3: How does the use of audio feedback impact the sense of community in ALN? 

• RQ 4: In what manner is perceived learning impacted by the use of audio feedback? 

• RQ 5 : What relationship exists between the use of audio feedback and student satisfaction? 


A. Instructional Setting 

Curriculum and Instruction 687, Advanced Teaching Strategies, was the course through which this study 
was conducted. Prior to this study, C&I 687 had been offered completely online for three consecutive 
semesters. 
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Structurally, C&I 687 consisted of ten learning units in which students explored and evaluated advanced 
teaching concepts and strategies. In the first unit, students were introduced to the philosophical 
foundations of constructivist teaching and asked to evaluate a series of readings with respect to their 
personal experiences in the classroom. In seven of the remaining units, students were introduced to eight 
teaching strategies (concept attainment, inductive learning, cooperative learning, synectics, direct 
instruction, mnemonics and classroom discussion) through readings that addressed methodology, through 
text and video based case study analysis and through discussion postings in which students were asked to 
apply the various models to content area lesson plans of their choice. Students then evaluated each other’s 
postings and refined lesson plan strategies based on the communal knowledge constructs that emerged. 


One of the two remaining units was a mid-term assessment activity where students selected two video- 
based classroom vignettes and conducted an evaluative case study for which they identified the teaching 
strategies employed, explained the usage rationale and suggested how the teacher might have improved 
the manner in which their students acquired knowledge. The final unit consisted of two parts: part 1 
consisted of six reflective activities in which students were asked to evaluate how praxis might be 
impacted by contemporary and emerging societal and technical issues; the second part of the final unit 
required groups of students to develop a series of thematic, interdisciplinary lesson plans in which 
strategies explored during the semester were utilized. These plans required that students use a minimum 
of three teaching strategies explored during the semester. After all projects were submitted, students were 
expected to evaluate plans submitted by other groups and suggest revisions. 


The course was a major elective for both master’s and doctoral level students in the Curriculum and 
Instruction program. The course had no prerequisites and was taken at various times during students’ plan 
of study. 

In previous years when this course was taught, feedback was provided to students in two ways. In the 
first, the instructor would interact with the students’ text based postings on the discussion board using 
Socratic questioning to enhance and expand upon various threads that emerged. Additional group 
feedback was provided at the conclusion of each thread. In the second, the instructor would provide 
individualized text based feedback via email to students on each discussion topic or submission. 


B. Use of Audio Commenting Within the Instructional Setting 

In addition to utilizing approaches to text-based feedback from previous years, we incorporated audio 
commenting in this iteration of the course. When posting audio comments to the discussion board, in 
emails to the entire class, or to small groups, the instructors produced wav files using Audacity freeware. 
The files were then added to the discussion board or email as attachments. 


In the case of individualized feedback, the instructors selected various discussion posts made by a student, 
copied them to a Word document, inserted comments and sent the document back to the student via 
course email. This type of individualized commenting was also used for the midterm case studies, final 
reflections and the group project. 


We provided approximately half of the individualized feedback in a text-based format and the other half 
via audio. At the end of the course all students had received six documents in which text feedback was 
used and five in which audio feedback was used. To avoid the introduction of bias, prior to the beginning 
of the semester each assignment was given a number from one to 12. These numbers were then entered 
into excel and randomized. From this list, we assigned alternating text-based or audio feedback as the 
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modality that would be used. 


On the discussion board, we engaged in Socratic questioning as in previous semesters. At the end of the 
semester, the discussion board contained a total of 1471 postings and replies. Of these, 203 were Socratic- 
type questions that we posed to students on an individual basis. In addition, we provided another 59 
postings that took the form of group feedback: 3 1 of these were text-based and 28 used audio. 


Technically, the audio feedback was produced by first copying select discussion board postings into a 
Word document or opening a Word document in which students had submitted individual assignments. 
The Word document was then converted into a PDF document using Adobe Acrobat Pro 7. Once in this 
format, the instructor used the Record Audio Comment tool within the Comment and Markup option. 
Depending on a host of factors, including length, number of topics discussed and quality of the work 
submitted, the instructor placed varying numbers of audio files within the document, as well as a 
summary statement at the end of each document. The audio feedback was spontaneous in nature, as it was 
intended to replicate the non-scripted verbal interactions that occur in F2F environments. 


In the instances where students received text feedback, it was in the form of a PDF document using the 
Note Tool selected from the Comment and Markup option. Text comments were placed at various points 
throughout the document and at the end, in a fashion mirroring that used in the audio feedback. The same 
document format and co mm ent placement strategies were used to ensure that any difference in 
perceptions of the commenting modality would not be influenced by these extraneous variables. 


To determine what impact using audio commenting had on time required to provide feedback, we 
maintained a log of the amount of time required to provide both text-based and audio feedback. During 
the analysis of data, we also compared the volume of audio and text based feedback that was provided to 
students. 


C. Participants 

West Virginia University’s Institutional Review Board approved the protocol for this study to ensure 
ethical treatment of all participants. For the semester in which this study occurred, enrollment consisted of 
26 master’s level students and 8 doctoral students. Of the 26 master’s students, 17 were practicing 
teachers and 9 pre-service teachers. Geographically, 29 of the students who took the course were located 
in West Virginia, 3 were located in Maryland, 1 in Alabama and 1 was on military deployment in 
Djibouti. 


An email was sent to all students during the last week of the course asking for volunteers to participate in 
post-course interviews. Seven doctoral students, 15 master’s level practicing teachers and 5 master’s level 
pre-service teachers volunteered to participate. 


D. Design 

A nested mixed methods design with both concurrent and sequential components was implemented [38]. 
We gave priority to the qualitative components nesting quantitative data within them in order to enrich 
our description of participants’ perceptions related to audio feedback [39]. Three separate sets of data 
were originally planned for triangulation during data analysis and interpretation: end of course survey 
data, post-course interview data and final projects. Unsolicited qualitative feedback generated throughout 
the semester, though not originally part of the research design, was added as a data set because it 
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contained rich and compelling data that could not be ignored. The end of course survey data included both 
qualitative and quantitative components collected concurrently with the final project data. Interview data 
gathered sequentially allowed us to follow up on themes generated from the end of course survey results. 


We selected a mixed methods research design for our work, and being guided by a “pragmatic approach” 
or paradigm [40] we sought to capitalize on the strengths of both quantitative and qualitative approaches 
to data collection. This clearly required following established criteria for generating high quality 
quantitative and qualitative data. While criteria for judging the quality of quantitative studies are well 
established, there is less agreement regarding what quality criteria are applicable to qualitative research 
[41, 42]. Searle [43] argued that triangulation of data sources aimed at enriching understanding through 
and of multiple perspectives should be the central criteria by which qualitative research is judged. Taking 
his point, we included multiple forms of qualitative data (survey, interview, and document), blended with 
quantitative (survey) and quantified (document) data, and analyzed these using strategies designed to 
achieve triangulation. 


1, Unsolicited Feedback 

During the semester, 14 students sent a total of 16 unsolicited emails to the instructor related to the use of 
audio feedback. The rich data in these emails provided early insight into how students perceived the 
modality, as well as technical difficulties that a small number of students were experiencing. The emails 
were coded and categorized based on thematic similarities that emerged in cross case analyses. Although 
this was not data originally designed into the study, this unanticipated feedback clearly added to our 
understanding of students’ perceptions of audio feedback. Capitalizing on the emergent nature of 
qualitative inquiry, this data set was included as an extra point of validation in the triangulation process. 


2, End of Course Survey Data 

At the end of the course, students were asked to complete a survey to assess satisfaction and perceived 
learning. The survey consisted of 52 items. The first 50, derived from instruments previously developed 
by Spencer and Thompson [44, 45], addressed student satisfaction with course design, perceived learning 
and sense of community. Two additional items related specifically to the use of audio feedback were 
added: 1) a Likert-type scale item addressing student perceptions of the relative effectiveness of audio 
versus text-based feedback, and 2) an open-ended item soliciting additional comments relative to audio 
feedback. The Likert-type item was analyzed using descriptive statistics. Responses to the open ended 
item were coded and thematically categorized using cross case analysis. This analysis then informed the 
semi-structured post course interview protocols. 


To guard against a novelty effect, as is often seen in student satisfaction with online courses [15, 46], we 
continued to collect data from other courses in which the instructors used audio feedback. This 
quantitative data consisted of responses to two questions. In the first, “I prefer audio feedback to text- 
based feedback,” students were asked to respond on a five point Likert-type scale with choices ranging 
from Strongly Disagree to Strongly Agree. The second question asked students how many courses they 
had previously taken in which audio commenting was used (0, 1, 2, 3, or more than 3). 


3, Post Course Semi-structured Individual Interviews 

Of the 34 students enrolled in the course, 27 volunteered to participate in post course interviews. These 
semi-structured interviews were conducted during the two weeks following the end of the semester. 
During interviews, individual students were asked their impression of both the course and each type of 
feedback using an interview protocol guide (see Appendix A) developed following principles described 
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by Berg [47] and Patton [48]. Two interviewers were involved in the process to ensure consistency. In- 
depth probing of responses was conducted on an individualized basis to draw out more detailed data 
related to why students perceived audio feedback to be more or less effective, as well as how it may have 
altered their perceptions of what it meant to be a participant in an asynchronous learning network. 
Interviews lasted approximately 50 minutes and were audio taped using a portable mp3 recorder. After all 
interviews were complete, transcriptions were generated for coding. The transcribed interview texts were 
analyzed following suggestions by both Strauss [49] and Tesch [50] using an interpretive, iterative 
approach with emphasis placed on drawing out thematic strands. Because of the data richness, both within 
and cross case analyses were utilized to more fully represent what occurred at both the individual level 
and as part of a group dynamic. 


To guard against a novelty effect, check for consistency in themes, and detect new themes, a total of 5 1 
students were randomly selected from 17 courses in which the instructor had used audio commenting 
since the completion of the original study. These students were emailed a questionnaire (Appendix C) in 
which they were asked to reply to a series of open ended questions. The questionnaire was derived from 
the interview protocol used to conduct the post-course interviews (Appendix A). Using an iterative, 
interpretive process, themes were drawn out in the same manner used for transcribing the original 
interviews. 


4, Final Project Document Analysis 

The final project for this course required groups of students to develop a series of thematic, 
interdisciplinary lesson plans that utilized a minimum of three strategies explored during the semester. 
Document analysis of final projects was conducted by first coding for the types of strategies students 
chose to use for lesson plan design and then categorizing based on the type of instructor feedback (text 
versus audio) used when students studied these strategies earlier in the course. The incidences of the 
various categories were quantified and descriptive statistics calculated to explore how feedback modality 
might have impacted content usage. 


The final projects were then recoded to determine the level of Bloom’s taxonomy [51] applied to each 
strategy. In this process, the lesson plans students developed were decompressed and individual activities 
evaluated using a rubric derived from Slavin’s [52] application of Bloom’s taxonomy to pedagogy (see 
Appendix B). Coded documents were reviewed by two researchers to ensure consistency. The reviewers 
unanimously agreed on the coding. The results were presented using descriptive statistics to determine if 
audio feedback impacted the level at which content was used. 


5. Triangulation 

After analyzing each data set in the manner described above, open coding was used to isolate prevalent 
themes followed by negative case analysis to explore consistency across data sources [53]. First, the 
results of the quantitative end of course survey question were compared with the findings from the post 
course interviews and unsolicited feedback for additional confirmation. Next, the findings from analyzing 
the qualitative question in the end of course survey were crosschecked with the interview data and 
unsolicited feedback. The end of course survey did not address content retention and so could not be 
crosschecked with the document analysis. Usage frequency and level counts derived from document 
analysis were checked for consistency with interview data focused on content retention. The interpretive 
conclusions from triangulation analyses were then compared to what is known about corresponding 
elements in learning theory and social presence literature to develop grounded theory that could be 
applied to future research. 
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III. RESULTS OF THE STUDY 

This study was originally designed with three data sources: end of course survey data, semi-structured 
interviews and document analysis. Flowever, data rich material in the form of unsolicited feedback from 
students was included as we found it to lend significant insight into what students thought at the time they 
were actually receiving the audio feedback. In addition, these unsolicited emails allowed us to refine 
some of the potential probing areas in the interview guide. Results of analyzing each data source 
separately are provided below. Triangulation, observations and conclusions follow in section IV. 


A. Unsolicited Feedback 

Fourteen students in the course sent a total of 16 unsolicited emails regarding the use of audio feedback. 
In 14 of these, 1 1 of which were sent within three days of the initial use of audio feedback, students wrote 
to express a high degree of satisfaction with the modality. The remaining two emails were related to 
technical problems with getting the audio files to play. No unsolicited emails expressing negative 
sentiments about the use of audio were received. 


The following is typical of the unsolicited emails: 

It is very rewarding and helpful to HEAR your comments. Now I understand more about what you 
are trying to say than I did with the last set of feedback we got. Thanks! 


In an email received about three weeks after audio commenting was first used a student offered the 
following: 

We’ve had written comments twice and verbal co mm ents twice now. Eet me guess — this is 
someone’s research project right? Eet me just save you some time. The verbal feedback is much, 
much, much better than the written. I said the same thing when I talked to you on campus last month. 
So can you just send me the voice co mm ents from here on out, say there is no comparison between 
the two at all and nix the written stuff? That’s probably not going to happen, but I thought it was 
worth a shot! 


B. End of Course Survey Data 

The end of course survey (response rate = 91%) included two audio feedback specific items: one 
quantitative Eikert-type item and one qualitative open-ended item. For the quantitative item, 26 of 3 1 
respondents indicated that they believed audio feedback was more effective than written feedback. Four 
believed there was no difference between the two modalities and one responded with a N/A. The N/A 
response was explained in the qualitative item as described below. 


When asked for additional comment related to the use of audio feedback, 1 1 students responded. Of these 
responses, 1 0 were highly positive and cited audio feedback as a primary reason for being satisfied with 
the course. 

I usually find online classes rather boring. That was not the case here. It was definitely because of the 
way the instructor communicated with us using the audio PDF’s. That approach made me interested 
for the first time in what was happening in an online class. I didn’t feel like I was just jumping 
through the hoops when I got to hear the comments on my work. 


No students provided negative comments related to the audio feedback. The response not categorized as 
positive addressed technical problems, clarifying the single N/A response to the quantitative item. 
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I would definitely take an online course again, but I hope I can get this audio thing worked out if that 
is the way we will get comments in other courses. Even after working with tech support I never could 
get the files to play on my home computer. I did get them to play at work though. Because of this 
issue I didn’t believe I could answer the question on audio commenting in the way it was intended 
and therefore said it was not applicable. 


After the course was over, this student contacted us regarding her technical problems. It was discovered 
that a broken sound card in her home computer was at fault. 


The survey data collected from other courses to address a potential novelty effect resulted in a 68% 
response rate. Of the 312 respondents, the mean number of previous courses with audio commenting was 
1.31 (SD = 1.29) with 99 students having at least two previous courses utilizing this feedback modality. 
Responses to “I prefer audio feedback to text-based feedback” averaged 4.46 (SD = 0.78) corresponding 
with halfway between strongly agree and agree. In fact, only 9 students out of 312 strongly disagreed or 
disagreed with the statement. Directly addressing any potential novelty effect, there was no significant 
relation between the number of courses students had experienced with audio commenting and their 
relative preference for that feedback method (Spearman Ts= .07, n = 312, ns). 


C. Semi-Structured Interviews 

Students indicated that they preferred audio feedback to written feedback in 25 of the 27 interviews. One 
student had no preference and one preferred written feedback. From the 25 students who preferred audio 
feedback, four general themes emerged: 1) increased ability to understand nuances that might be lost in 
written communication, 2) feeling more involved in the course, 3) improved retention of content and 4) a 
belief that the instructor cared more about the student’s learning. The mean number of themes expressed 
per interviewee was 2.28 (SD = 0.79). 


1, Ability to Understand Nuance 

The most frequently expressed theme (n = 19) was the ability to detect nuance and inflection in the audio 
commenting. In general, students believed that verbal feedback gave them increased insight into what the 
instructor was attempting to convey and that it produced a more comfortable, less formal learning 
environment. 


This perspective is best illustrated by one student who said: 

I have taken a couple of online classes and every time I would get these notes or critiques or 
comments back from the instructor and I would be wondering exactly what they were trying to say. I 
mean, I would understand what they were saying but not the way they were trying to say it. 
Sometimes you would wonder if they were agreeing with you or trying to figure out how to politely 
say you had it all wrong. 


Now, when I first heard the audio feedback I was like wow! I get what he is saying to me. It was all in 
your voice and I understood when you were saying something like well this is good, but 


I understood then that you really liked what I was doing but were trying to tell me to add a little more, 
but in a good way. Now, in the first time we got feedback it was written and you said some things that 
were kind of the same but I thought you were really trying to bust me for not doing a good enough, 
you know, job. Then I looked at my grade and it was good so I couldn’t understand exactly what you 
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were thinking. 


Was my work not so good and you just gave me a decent grade? Or was it ok and I just didn’t 
understand what [was being] said to me. When I heard you say something similar though the whole 
thing made sense. 


One student, who had some online teaching experience, took an analytical approach to introspection as 
revealed by the following: 

To answer what I think about this I need to tell you what I did. I’ve taught one online class for my 
department. . . well two if you count the one I am just finishing, so obviously I was fascinated when I 
got the first audio files along with my work. But I didn’t want to just jump on it because it was 
something new. What I did was sit down and transcribe what you sent over and then I looked at it. I 
looked at it and listened to the files again and kept doing this for a while. What I realized was that its 
two completely different things. 


I know you were saying the same things in your [audio files] and in what I transcribed, but the 
difference was you were saying them. When I looked at the transcription there was no stress placed 
on any of the words or sentences. Then I tried putting the stress there by adding in caps or 
exclamation marks and I wondered if I would have thought that you might have been yelling or 
something if I would have read it that way. What I figured out was that there is really no way that you 
could have gotten the same info across the same way. 


This all made me think about the way my students have perceived me in courses when I write to them 
with comments. It’s not the same is it? No, it’s really not. We lose so much in the written word 
sometimes and I think maybe we haven’t thought about that enough in our online teaching. [Online 
courses] are going to become ever more, uhm, you know, prevalent for all types of learners and I 
think we really need to figure out the best way to get our intent across. I think this is probably a really 
good first step. I know there are some things coming down the line that will make this look like we 
are taking baby steps, but they are steps I think we need to start taking so we can keep moving in the 
right direction. In a direction where we don’t get dehumanized and our students don’t lose what we 
are trying to get to them. . . or the way we are trying to get it to them. 


2, Feelings of Increased Involvement 

The belief that audio feedback increased feelings of being more involved and “a real part” of the class 
was the second most commonly expressed theme (n = 15). Though students often began their discussions 
of involvement in general terms, subsequent probing revealed that this perception was usually related to 
what they believed to be a lessening of social distance when audio was used. 


The richest data related to this perception came from a student who cited her feeling of being more 
involved as the primary reason for preferring audio feedback. Her response was as follows: 

Yes, I would have to say that audio [commenting] made all the difference in the world to me. I’ve 
taken several online classes here and at [another university] because they are so much more... uhm, 
easier for me to get to. The downside is that I have felt like I am the girl in the bubble. Some of the 
instructors have done these things like the biography postings and online groups that help you meet 
other students and get to know them; some haven’t. But even where they have [used these types of 
activities] you still feel like you are at home in your own little bubble and you are telegraphing out to 
all these other bubbles that other people are sitting in. Then between all of you there is this cold wall 
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type thing. It’s the course, the technology, all of that stuff that makes the course. There is this barrier 
there. 


Now, some of that has went away a little when we did things like be in chats, but it’s still all kind of 
unreal you know? Being an Art teacher and having done my undergrad at a [very liberal college] I 
suppose I’ve always been one to seek out some of that personal interaction. So, because of that I’ve 
always felt that these online classes are a little, you know, dehumanizing. 


That said, I get this file where you put in this audio and boom! It was all a big change for me you 
know? It was like that bubble started getting popped in all these different places and made me feel 
like you were reaching in there and touching me. I know that’s probably kind of silly, but just your 
voice alone made me feel like it was a real class and not this big technology construct that was 
locking us into its parts. 


This really changed the way I viewed the whole online learning thing. I know we aren’t looking at 
learning the way that Judy Jetson might be learning but this tells me that we are moving that way. We 
are starting to reach out to each other across our phone lines and I think that’s really important you 
know? I wish we could be doing this with each other as well as just you sending us these clip things 
and all. Like when we did our group projects, if we could have talked to each other like this it would 
have been a whole Brave New World thing going on between us but in a really good way. 


Guys, keep doing this kind of stuff Next semester and I’m done with my masters and I didn’t know if 
I would every take another online class or not, but if I could see a class where this was going on 
between me and the instructor and me and the other [students] then I would be all about learning this 
way. 

Another student who cited feelings of increased involvement was less eloquent in her initial response 
when she simply answered: 

The audio, well, I also like it because it makes me feel like a real part of the class. You don’t feel like 
a number when you get that. 


However, subsequent probing revealed much more about her perceptions: 

Here’s the thing, we get all these written comments back and they are all really dense and dry. At 
least they seem dry. This goes back to what I meant about the inflection in the instructor’s voice. 
When you get this written feedback it could be something where maybe the instructor has taught this 
course lots of times before and has all of these canned responses ready on a Word file and just cuts 
and pastes them into our work to save all [of their] time. I know that’s probably not what’s going on, 
at least I hope it’s not, but sometimes you can feel that way. You feel you might have a robot 
responding to you. 


What’s different though with the audio though is that you know that its not canned. It could even be 
the same comments, but the delivery makes you feel like you are part of this learning group and that 
makes it all good. It makes you want to be involved, because you have this involvement level that is 
going to be coming back at you. 
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3. Content Retention 

For students (n = 12) who cited increased learning and content retention as reasons for preferring audio to 
written feedback, most (n = 9) related their preference to learning style. The following is typical of 
students in this category: 

I think the reason I like the comments made with the audio thing is that I learn better that way. Let’s 
take when I’m in a lecture class. I look around and everyone takes all these notes but I set there and 
listen and record what’s being said. Then when I’m studying I listen to the recording over again. I just 
retain better that way. With this feedback its just an extension of that; the audio I retain the first time, 
the written I might read four or five times. 


For the remaining three students who cited increased retention with audio feedback, the following is 
representative: 

I like this [audio feedback] because I am listening to what you are saying and scanning what I wrote. I 
can see what you are talking about and it clicks that way. Now, granted, I might have to listen to it 
and read it two or three times because doing both at once makes it all not stick as well, but in the end 
it works better than if both parts had been written only. 


Interestingly, no students expressed a dislike for audio feedback because of learning styles. Flowever, four 
did express views similar to the following: 

What I find... well odd, is that I’ve taken learning style inventories and I know that I am very, very 
visual. Based on that you would think that I wouldn’t like this type of feedback at all. I know that I 
should be liking the written comments much more, but that wasn’t the case. I can read comments 
once and I remember. Flere I was listening twice, sometimes three times to what you said to make 
sense of it all. Flowever, it goes back to what we talked about earlier about feeling like I was part of 
the class, a real part. That offset by far the whole learning styles issue. I guess its like when we are in 
the classroom, we feel like the teacher is telling us something and bringing us into a discussion so we 
don’t expect them to write it too. Maybe that’s what’s going on here. Maybe because you made me 
feel more like I was part of the class I didn’t feel like I necessarily needed everything presented in the 
way that I learn the best. 

4, Instructor Caring 

The final theme expressed by students (n = 10) was related to the degree they perceived the instructor to 
care about their learning when audio versus written feedback was provided. In most instances (n = 8), this 
perception was closely associated with nuance and feelings of involvement as eloquently expressed by 
one student when she said: 

The final thing is about the way I think the audio shows that you cared about us. It’s not really 
something that’s out there by itself though so I need to talk about the whole picture if that’s alright 
with you? 


I started talking about all of this by talking about feeling the tone of your voice and knowing more 
about what you were trying to say than when I got just the words on paper. . . err rather on screen. . . 
well whatever. We can start there and then when I got to understand what you were saying it gave me 
some idea of who you were and that made me want to be more involved. Then when I started feeling 
really involved and all it made me feel like you really cared about what was going on. That’s a warm 
fuzzy I haven’t gotten with online classes before. 
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A similar sentiment, though expressed quite differently, was provided by another student who said: 

You took the time to try out this new audio file thing and actually communicate with us. Earlier I told 
you how I thought that it was way better than just reading words that might be misunderstood. That’s 
true and so is the part when I said it made those connections that brought the class together. But what 
I left out is that it also showed that you were interested in our, in us learning what was going on. 
When you take the time to establish something that’s this complex it shows you want us to really be a 
class and not just a group of individuals all doing something similar. I know teaching is pretty 
thankless, but I do want you to know that I appreciate what went on this semester. I can’t really say 
that I’ve said that about any of my other online classes, but you talking to me, I mean really talking to 
me, and everything that was built up from that, made me feel that way here. 


From the 5 1 questionnaires ( 1 00% return rate) sent to students in 1 7 other courses where audio feedback 
was used in order to address any potential novelty effect, the same themes emerged with slightly different 
weighting than in the original study. No new themes were revealed. The prevalence of themes is 
presented in the following table: 


Theme 

Prevalence 

Ability to Understand Nuance 

42 

Feelings of Increased Involvement 

26 

Content Retention 

27 

Instructor Caring 

32 


Table 2. Prevalence of Themes in Follow-Up Questionnaires 


D. Document Analysis 

Final projects were analyzed in terms of relative usage of strategies for which audio or text feedback was 
provided. Two measures were used in this process to assess both frequency and level of use. 


The assignment required students to use a minimum of three strategies that had been covered during the 
semester in completing their final project. The mean number of strategies used across five groups was 4.2 
(SD = 1.09). The number of strategies incorporated into final projects after having received audio versus 
written feedback is provided in Table 3. 



Total Number of 
Strategies 

Strategies for Which 
Audio Feedback Was 
Received 

Strategies for Which 
Written Feedback Was 
Received 

Group 1 

4 

3 

1 

Group 2 

4 

4 

0 

Group 3 

3 

1 

2 

Group 4 

4 

3 

1 

Group 5 

6 

4 

2 


Table 3. Comparison of Strategies Used in Final Projects by Feedback Type Received 


Coding of documents revealed that students were far more likely to apply higher order thinking and 
problem solving skills (Synthesis and Evaluation in Bloom’s Taxonomy) to content for which they had 
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received audio feedback. Table 4 depicts the level at which strategies were applied in final projects 
disaggregated by the type of feedback received for those strategies. 



Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluation 

Written 

1 

2 

1 

1 

0 

1 

Audio 

1 

1 

2 

0 

5 

6 


Table 4. Comparison of Level of Application by Feedback Type Received 


E. Comparison Time Requirements and Quantity of Feedback for 
Audio vs. Text 

During the course of the semester, 204 documents containing text-based feedback and 170 containing 
audio feedback were generated. The mean feedback volume for text feedback was 129.75 words (SD = 
57.43) and 331.39 (SD = 89.31) for audio. The mean time required for the instructor to provide feedback, 
was 13.43 minutes (SD = 4.53) for text-based feedback and 3.81 minutes (SD = 0.76) for audio. The time 
required to read the documents prior to / during commenting did not differ significantly as a function of 
the feedback modality used. The mean time for reading the documents when text-based feedback was 
used was 14.13 minutes (SD = 5.45) and 13.94 minutes (SD = 5.74) when audio feedback was used. The 
average file size for audio feedback was 258 kb / min (SD = 23.21). 


IV. DISCUSSION AND CONCLUSIONS 

Our investigations revealed an overwhelming student preference for asynchronous audio feedback as 
compared to traditional text based feedback, with no negative perceptions of the technique. The fact that 
over one third of students cited the use of audio feedback as a key factor they would use in selecting 
future online courses is significant. When these findings are combined with data comparing the use of 
knowledge constructed using audio feedback and the level at which that knowledge was applied, we 
believe asynchronous audio commenting merits serious consideration in the development and delivery of 
future courses. 


Though students can project themselves and their emotions through text based communication [15, 16, 
20, 21], two thirds of students (n = 19) in this study cited ability to understand nuance as reason for 
preferring audio to text feedback. This finding is important because it extends upon Richardson and 
Swan’s [22] social presence research, in which a strong relation (R^ = 0.36) was found to exist between 
students’ perceptions of social presence and satisfaction with the instructor. In addition, it is likely that an 
enhanced ability to detect nuance impacts student perceptions of the instructor’s use of humor, and 
openness toward and encouragement of student ideas and discussion; key immediacy behaviors cited by 
Arbaugh [23]. 


The second most commonly expressed theme, increased feelings of involvement, is important because it 
reinforces the sense of community and perception of “being there.” In terms of how audio commenting 
decreased social distance for students, the best example can be found in words offered by one student: 

It was like that bubble started getting popped in all these different places and made me feel like you 
were reaching in there and touching me. 


We consider the role audio feedback played in developing this type of interpersonal relationship with 
students in our asynchronous courses to be a compelling enough reason for its continued use even if no 
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other positive factors had been discovered. 


Findings related to perceptions of increased caring on the part of the instructor, a theme that was 
frequently tied to nuance and increased involvement, confirm opinions held by Olsen [3 1 ] from his use of 
the technique in the traditional classroom. Though students were hesitant to explore this theme in great 
detail during the initial interviews, it was apparent that it was of considerable importance and increased 
overall satisfaction with the course and the instructor. The significant increase in the percentage of 
students expressing this theme in follow-up questionnaires in subsequent courses is worth noting. We 
believe that audio feedback should be considered a means by which to increase positive perceptions of the 
quality of instructor interactions and, by extension, social presence in ALN. 


While the preceding three themes support our contention that asynchronous audio feedback increased 
teaching presence and decreased social distance, it may be even more important to examine the positive 
impact the technique had on perceived learning. Though slightly less than half of all respondents, in both 
the original and follow-up interviews, indicated that they retained information and were able to synthesize 
instructor comments better when they received audio feedback, document analysis in the original study 
indicated that the impact may have been even greater. 


Random assignment was used to determine whether audio or text feedback was utilized for each topic and 
our analysis revealed no differences in difficulty for topics assigned to each type of feedback. Even given 
that control, information for which audio feedback was provided was used approximately 350% more 
frequently than information for which text based feedback was provided. With respect to level of 
application, students applied content for which audio feedback was provided at the two highest levels of 
Bloom’s Taxonomy in slightly more than 70% of the cases. In contrast, content for which text based 
feedback was provided was only explored at similar levels in less than 20% of cases (see Table 2). Not 
only did students retain material better when they received audio commenting on it, but they applied that 
content in more cognitively complex ways. 


These findings indicate that audio feedback enhanced learning for our students; though much more 
research needs to be conducted to determine how generalizable these finding may be across subject 
matter, instructors, and institutional contexts. Since the completion of this study, other early adopters in 
our College have experimented with audio feedback following the techniques we employed. The 
quantitative, qualitative and anecdotal evidence has been overwhelmingly positive. Over 450 students in 
courses taught by these instructors have now received audio feedback. According to these instructors, 
approximately one third of their students have submitted unsolicited feedback expressing a strong 
preference for this technique over text based feedback. No negative feedback has been received. 


From the instructors’ perspective, the ability to reduce the time required to provide feedback by 
approximately 75% was a compelling reason to adopt the technique. However, it is important to note that 
this reduction in time was coupled with a 255% increase in the quantity of feedback provided. While 
increases in quantity of feedback delivered with less demand on instructors’ time is a strong reason to use 
the technique, evidence that it also increased retention and understanding of content at deeper levels 
makes it hard to argue against using audio commenting at this point. Still, more research is needed to 
determine potential differences in the types of feedback provided when text-based and audio feedback are 
used, and the precise mechanisms that facilitate increases in student learning. 
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VII. APPENDIX A 


Interview Protocol Guide 

Good moming/aftemoon/evening. The goal of this study is to examine some of your observations related 
to the course you have just completed, C&I 687, and the auditory feedback mechanisms that were used. 
The information generated by the study will be used in a research project that is designed to benefit both 
students and faculty with respect to the use of this medium. With your permission, I would like to 
audiotape this interview. 


Before we begin, I would like to notify you of the following: 

• Y our participation is entirely voluntary. Y ou may halt the interview at any time and/or choose not 
to answer certain questions. 

• Your responses will remain anonymous. Complete confidentiality will be maintained. At no time 
will your identity be revealed either by the procedures of the study or during reporting of the 
results. 

• No negative consequence will result for choosing not to participate. 


Please feel free to tell us what you really think and feel; this will be the most helpful in trying to find out 
how to improve things for students and faculty members in the future. 


Thank you for your participation in this research. 


[Note code number and start recording.] 

1. What was your overall perception of C&I 687? 

(probe for each one: 1. likes and dislikes 2. time required to complete assignments) 

2. How did the course compare with traditional courses you have taken? 

(probe for: 1. activity types 2. interaction) 

3. How did the course compare with other online courses you have taken (if any)? 

(probe for differences as needed) 


4. How effective, in your experience, is online learning as opposed to fZf? 

(probe for: 1 . quality of discussion 2. quality of products 3. quality of interaction 4. other concerns) 


5. What did you think of the types of feedback used in the course? 

(probe for individual versus group responses and auditory versus written media) 


6. When you think about the auditory feedback that was used, how would you describe your reaction to 
the instructor comments as opposed to written feedback? 

(probe as needed) 

7. Do you think that auditory feedback is more or less personal than written feedback? 
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(probe as needed) 

8. Other than what we have discussed, what did you like or dislike about auditory feedback? 

(probe as needed) 

9. Are there any ways in which you believe that audio feedback impacted your ability to construct 
knowledge in this course? 

(probe as needed) 

10. That is all I have. Is there anything else you would like to add? 

Thank you for participating. 


VIII. APPENDIX B 


Final Project Rubric 

Knowledge: Students explain the step-wise procedures for delivering instruction using a specific teaching 
strategy. Syntax is in the appropriate order; however, there is no elaboration on the methodology 
employed. 

Comprehension: Students expand on the syntax of various teaching strategies by describing the model, 
as it is applied to their lesson plans, by explaining key concepts, predicting outcomes or identifying key 
issues that influence student learning. 


Application: Students clearly apply their knowledge of teaching strategies to the content area; defined as 
content pedagogy. 


Analysis: Students break down lesson plans into component parts and analyze the strategies employed. 
As an example a student would match the syntax of a given teaching strategy to the goals and objectives 
of activity. 


Synthesis: Students apply prior knowledge from content and curriculum studies to the teaching strategy. 
Indicators will include inclusion of modifications to the primary teaching strategy that require the 
inclusion of innovative designs or combining multiple strategies into a single construct. 


Evaluation: Students include, in their lesson plans, discussion elements in which judgments are made and 
justified by the inclusion of a set of criteria. Terminology such as compare, summarize, decide and asses 
are likely to be present in such discussions. 


IX. APPENDIX C 
Student Satisfaction Questionnaire 

The goal of this study is to examine some of your observations related to the course you have just 
completed and the auditory feedback mechanisms that were used. The information generated by the study 
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will be used in a research project that is designed to benefit both students and faculty with respect to the 
use of this medium. 


Before you complete the survey please be aware of the following: 

• Your participation is entirely voluntary. You may choose to answer or not answer any or all 
questions. 

• Your responses will remain anonymous. Complete confidentiality will be maintained. At no 
time will your identity be revealed either by the procedures of the study or during reporting of the 
results. 

• No negative consequence will result for choosing not to participate. 


Please feel free to tell us what you really think and feel; this will be the most helpful in trying to find out 
how to improve things for students and faculty members in the future. 


Thank you, in advance, for participating in this study. 


1 . What was your overall perception of (course name and number here)? Please describe what you liked 
and disliked about the course. 


2. Flow did the course compare with traditional courses you have taken? When answering this question 
think about the types of activities, interaction with the instructor and interaction with fellow students. 


3. Flow did the course compare with other online courses you have taken (if any)? Please elaborate a little 
on differences (either positive or negative). 


4. Flow effective, in your experience, is online learning as opposed to fZf? If, in your opinion, the 
following are applicable, please elaborate: 1. Quality of discussion. 2. Quality of learning. 3. Quality of 
interaction. 4. Any other issues you care to discuss. 

5. What did you think of the types of feedback used in the course? 

6. When you think about the auditory feedback that was used, how would you describe your reaction to 
the instructor co mm ents as opposed to written feedback you may have received in this course or previous 
courses? 

7. Do you think that auditory feedback is more or less personal than written feedback? Why? 


8. Other than what we have discussed, what did you like or dislike about auditory feedback? 


9. Do you have any other comments about the course or the instructor? 

When you have completed the survey please save it as a Word document and email it to (insert email drop 
here). Thank you once again, for agreeing to complete this survey. 
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